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Abstract. - The stability of the TAP mean field equations is reanalyzed with the conclusion 
that the exclusive reason for the breakdown at the spin glass instability is an inconsistency for 
the value of the local susceptibility. A new alternative approach leads to modified equations 
which are in complete agreement with the original ones above the instability. Essentially altered 
results below the instability are presented and the consequences for the dynamical mean field 
equations are discussed. 



Introduction. ~ Together with the repUca approach, the Thouless-Anderson-Palmer 
(TAP) approach Q is the most important method to analyze infinite range spin glass models 
like the Sherrington-Kirkpatrick (SK) model of Ising spins (for reviews see ^)- The 
TAP equations are well established [^j-^ and are exact in the thermodynamic limit {N — > oo) 
provided that the local magnetizations {mi} satisfy the condition 

a; = 1-/3^(1 -2^2 + '74) >0 where = N-^^rrf^ i^ = 2,4. (1) 

i 

The condition (|l|) represents the central stability condition for the SK spin glass and therefore 
is also found in other approaches |^,|^. Within the TAP approach two different arguments 
for (|^) are known. Bray and Moore [/[found a divergence of the spinglass susceptibility for 
X 0. The expansion of Plefka ||], leading to the TAP equations, is limited to a; > 0. 

In next section a further, basically simple, aspect of the stability condition is presented, 
leading naturally to a modification of the TAP equations. 

Stability analysis revised. ~ The TAP free energy F of the SK model is given by 
F(/3, {mi}) X! - ^ 92)^ - y^fejWi 

ij i 

i 

where the {hi} are local external fields. The bonds Jij are independent random variables with 
zero means and standard deviations N~^^^ . From dF/drrii = the TAP equations 

m.i = tanh/3|/ii + ^ Jijinj - l3mi{l - 92)| (3) 



© EDP Sciences 



2 



EUROPHYSICS LETTERS 



result and the stability is governed by the inverse susceptibility matrix 

X',^ = dKldm, = d'^Fldm^dm, = <5,,{/3-i(l - m\)-^ + /3(1 - 92)} - J^, (4) 

where the iV~^- order term —2N~^ Pmiirij has been dropped as it has no influence on the 
present analysis (compare, however, and the footnote below). 

According to (^, the free energy is formally well defined for all values of the {rrii} inside the 
hyper cube \mi\ < 1 including the regime a; < 0. The question is which physical requirement 
is violated for x < 0. To answer this question we investigate the free energy for convexity, a 
fundamental requirement for every free energy function. This can be done without explicit 
knowledge of the solutions of (||). The analysis is performed analogously to |^ and the 
resolvent of the matrix is introduced 

R{z) = N-'TT{z~x''r'- (5) 

It is important to note (compare appendix) that R{z) is an element of the class of functions 
which are analytic in z for Imz ^ and such that 

Im R{z) > for Im z < . (6) 

The key for further analysis is the powerful theorem of Pastur 1^, rederived in [Q as ' 
locator expansion' . According to (|2^ ) the resolvent R{z) satisfies the equation 



i?(z)=iV-i^{z-i?(z)-/3-i(l-TO2)-i_;3(l_q2)} ' (7) 

i 

in the iV — > 00 limit for nearly all configurations of the bonds (which does not imply averaging) . 
Eq. (0) is basically a polynomial of the order N + 1 and has therefore iV + 1 solutions for 
R{z) . With reference to the appendix and to however, there is only one solution which 
satisfies (^. Thus R{z) can uniquely be determined. 

Near x = and z — 0, the leading behavior of R{z) is obtained by a power expansion of 
the right hand site of (^ in terms of {R — z + /3{1 — (72)} • To second order 

p {R ~ z + (3{l - q2)Y + X {R ~ z + f3{l ^ q^)} + z ^ (8) 

is found, where p = P^N^^ ~ ^f)^ is a positive quantity. For the case p\z\ ^ x'^, the 

solutions R^{z) are given by 

{R+ - z + l3{l - q2)} = -z/x and by {R^ - z + l3{l - q2)} = -x/p + z/x (9) 

where according to the requirement (|^) the solution R'^{z) applies for x > and the solution 
R~{z) applies for a; < 0, respectively. 

Several important conclusions can be made. According to (|^) 

Xi = -ReR{z) ^N-^Tix (10) 

2:^0 

holds for the local susceptibility xi : which gives, with eq. (||) 

X;=/3(l-q2) for a; > and xi ^ - q2) + x/p for a; < , (11) 



respectively . The value of xi deviates from the value /3(1 — 92) for x < 0. The latter value, 
however, is exact for an arbitrary Ising model in the canonical distribution as can easily be 
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shown. Moreover, in general, the exact value is used at the beginning of the derivations leading 
to the TAP equations. Thus the inconsistent result xi 7^ — 92) causes the breakdown of 
the TAP approach for x < 0. 

The spin glass susceptibility Xsg is, according to (||), related to R{z) by 

X.,^-Re^ ^N-'Trx' (12) 

from which in leading order 

Xsg = \X\~' [X ^ 0) (13) 

is obtained. Apart from the divergence, Xsg is well behaved and positive everywhere. The 
earlier work is therefore restricted to the case a; > 0, although this is not explicitly stated. 
Within the replica method Xsg becomes negative for a; < 1^,^. According to (||), a negative 
Xsg only results if ^ is violated. 

The eigenvalue density q[X) of is determined from R{z) by 

e{\)^n-HniR{\-ie)l^^^ . (14) 

With the full solution of the quadratic eq.(|^) one obtains for small values of A and of x 

g(A) = TT- V^i/VA - a^VP- (15) 
This result shows that the minimum eigenvalue of is given to leading order by 

A™„ = x'/p > for \x\ ^ 0. (16) 

With the extension to arbitrary values of x given in the appendix, the last result implies that 
negative eigenvalues do not occur and the TAP free energy is semi-convex everywhere. Note 
that the present analysis and thus the results are limited to the leading order in N . Hence 
the TAP free energy may not be semi-convex everywhere on sub-extensive scales. Such an 
interesting behavior was found for the p-spin spherical model and is expected in regions 
near x — Q due to finite size effects of q{\). |(^ 

The modified TAP equations. - It was shown that the TAP equations break down for 
a; < due to an inconsistency for the value of xi- We therefore look for modified equations 
which do not show such an unsatisfactory behavior. The present analysis starts with the well 
founded expressions [0,^,^ which are rederived in the appendix 

mi = tanh/3{ft,i + ^ Ji^TTij - m^x;} (17) 
j 

where the local susceptibility is given by xi — ^^^J2iXii — N^^^^dnii/dhi. With the 
working hypothesis that dxi/drui is a order N^^ term, the inverse susceptibility matrix x^j^ = 
dhi/druj is calculated from eqs.(|l^ to 

X--' = S^dr\l ~ ni^r' + Xi} - (18) 

(^)In this context the sub-extensive ternas to neglected in oq. and leading to a sCjCand stability 

condition 1 — 20^ {q2 — Qi) > (compare [H|) may have some effect despite the conclusions of llOl that these 
terms are not important. 



4 



EUROPHYSICS LETTERS 



Note that the eg5.([r^),(p^ and ^jXijXjk ^ ^ii^ represent a closed set of self consistent 
equations for the rrii, the Xij ^nd the Xij^- This fact is the key to the present approach. 
Moreover it is important to reahze that already the eqs.(p^ are complete and sufficient to 
determine all solutions as the eqs.(^8|) result from eqs.(^^. In this context the restriction of 
the original TAP equations to a; > is obvious as the replacement xi ^ ~ Q2) is only 
justified for stable solutions but will lead to inconsistencies for other solutions. 

The theorem of Pastur leads to a very effective simplification of the above set of equations. 
Application of this theorem yields R{—ie) = {/^"^ (1 — m^)^^+ R{~ie) +xi+^^} 

where R{z) is now related to the given in eq. ([l8|) . Performing the limit e +0, taking 
again care of the requirement (||), noting that xi = ^Rei?(— ze)|£^_)_o holds and separating 
the real and imaginary parts finally results in 

Xi=/3(l-g2) , r = for2;>0 (19) 

~ TV ^ 1 +r2/32(l _ ^2)2 ' N l+r2/32(l-mj)2 lorx^u^zuj 

where F = 7r^^p(0) ~ Imi?(— «e)|e^+o > was introduced. Note that both xi and F are 
continuous at x and note the latter eq. of ( po|) has always a solution F for a; < 0. Finally it 
is easy to show that the above working hypothesis is satisfied. 

The set of eqs.(|7|), ( |l9| ) and ( po|) represent the modified TAP equations which determine 
{rui}, xi F for all temperatures and all magnetic fields. In the region a; > 0, the modified 
and the original TAP equations and thus their solutions are identical. Dramatic changes, 
however, are found for the region x < in which the equations differ, leading in general 
to different solutions. The stability of the solutions is governed by the value of F which is 
proportional to the eigenvalue density p{Q) . In the region a; > all solutions are stable with 
F = and in the regime x < all solutions are unstable with F > 0. 

A solution of special interest is the paramagnetic solution = in zero field. This 
solution satisfies the modified TAP equations with F = (1 — T^y/^ > for T < 1 and with 
F = for T > 1, respectively. Thus the solution is unstable for temperatures below T = 1. 
This physically important result is not found by the original TAP approach which gives a 
stable paramagnetic solution for all temperatures. 

Eq. ( |l9|) shows that for all stable solutions the value of xi equals the thermodynamic value 
/3(1 — q2). A difference of these two values occurs for the unstable solutions. This is not in 
conflict with thermodynamics which is a priori limited to stable states. 

The last argument, the general limitation of equilibrium thermodynamics to stable states, 
implies that for unstable states thermodynamic functions like the entropy or the free energy 



are not defined [2} . Again from the strict thermodynamic point of view the original and the 
modified TAP equations are equivalent as both lead to the same results for the stable states. 
It is the reason for the breakdown which differs in both approaches. The inconsistency for xi 
for a; < of the original TAP equations is rather unconventional compared to the modified 
equations where the usual behavior, negative eigenvalues of applies. 

Unstable states are important for phenomenological extensions to dynamics. On basis of 
the modified TAP equations and in contrast to the usual TAP treatment such a phenomeno- 
logical but consistent extension to the dynamics becomes possible. In analogy to the 
relaxational Glauber dynamics in mean field approximation will be used for this purpose. 

(^) Discussions based on free energy landscapes use often extensions of thermodynamic free energies to the 
instable regimes. Thus such approaches are phenomenological. For the present work such an obvious extension 
is not available and the basically equivalent approach via relaxational equations of motions is employed. 
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Fig. 1 - Zero field solution of the modified TAP equations, (a) Edwards Anderson parameter 
52 (full line), x = 1 —0^(1 — 2q2 +g4)(dotted line), F (dashed line), (b) free energy density / — F/N 
(full line) and entropy density s (dashed line) versus temperature T for a system with = 100 and 
with Jij — ±N~'''^^ . The solution is obtained from the integration of ( |2l| ) by slowly cooling down 
from r = 1. The discontinuities at T = .078 and T — .551 result from saddle-node bifurcations 
with jumps to lower values of / (which are not resolved on the scale of (b)). Slow heating leads to 
hysteresis effects near the discontinuities. 



Measuring the time in units of the relaxation time, the equations of motion are given by 

mi{t) = -m,(t) + tanhP {hi + ^ Jijm,{t) - mi{t)xi{t)] (21) 

j 

where x/(^) again is determined by eqs.(p^) and (pO[). The fixpoints of ( pT| ) coincide with the 
solutions of static equations and a one-to-one correspondence between the dynamical stability 
of the fixpoints and the thermodynamic stability holds. Moreover the eqs.(pT|) simulate the 
evolution of thermodynamic processes in which [3 or hi are changed. Using the initial mag- 
netizations of such a process as initial values for the time integration, the system relaxes to a 
stable fixpoint which corresponds to a stable solution of the static TAP equations and thus to 
the final state of the thermodynamic process. The dynamical approach uniquely determines 
the final state even in those cases where the system exhibits meta- or multi-stability. From the 
pure knowledge of the static solutions such a determination of the final state is not possible. 

The final states are stable and thus they are always located outside the regime x < 0. 
During the dynamic evolution, however, the system may temporarily stay or may even start 
in the region x{t) < 0. Thus the flow of eq.(pT|) is needed in both regions x{t) > and x{t) < 0. 
These arguments demonstrate the relevance of the unstable states. Note that equations of 
motion of the above type but based on the original TAP equations usually lead to incorrect 
results in the spinglass regime as the system relaxes to paramagnetic solutions. 

As the eqs.(^) are phenomenological they do not describe the true dynamical effects of 
the SK model but can be used as a numerical tool to find explicitly the solutions of the 
TAP equations for finite A^. Numerical work which is based on this method and which is an 
alternative approach to |Q , will be published separately |jl^ . 

To demonstrate that this method is successful some results of 1 13| are presented in fig.(|^). A 



system of A^ = 100 spins with hi = and with binary distributions of the Jij was investigated. 
By numerical integration of eqs.(pT|) with (^9|) and (^) the fixpoints of these equations have 
been calculated in the temperature range from T — 1 down to T = .01 with a step size of 
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6T — .01. The first run at T = 1 has performed with the initial values rrii — 0. All other runs 
use as initial values the fixpoint values of the former run to simulate a slowly cooling down. 
The results for q2,x,V, the free energy and the entropy density are plotted in fig.(^. Due 
to finite size effects |7, 12ln3] the boundary for the stability x = is not exact for finite N . 



Thus as shown in fig.(l) stable fixpoint solutions with F > and with a; < are found. With 
increasing N, however, the boundary for stability tends to x = as demonstrated in jl^ . 

Conclusions. ~ Based on a careful stability analysis it has been worked out that the TAP 
equations need a modification in the region of instability. This modification leads to a complete 
and consistent description of the spin glass instability including the instable region. Although 
the new, unstable states cannot be observed, the unstable regime essentially determines the 
flow of the equation of motion and thus affects the basins of attraction of the stable states. 

The semi- convexity of the TAP free energy in the thermodynamic limit represents a further 
result of importance as it implies that the 'multi- valley' structure of the TAP free energy occurs 
on a sub-extensive scale. For further investigations of these important effects the modified 
equations are expected to be an adequate tool. It is straightforward to extend the analysis 
from the SK model to the other numerous spin glass models of infinite range and results 
similar to those of the present work are expected. 



Appendix: . - In the first point in this section eq.(|l7|), which is the starting point for the 
derivation to the modified TAP equations is rederived. Similar to the approaches [Q,^ the 
A'^ — 1 spin system is considered, with the Ising spin Sn removed from the N spin system under 
investigation. Taking the trace over the spin S'„ leads to the exact relations for all i{^ n) 

m — rh- + ^ ~^ ™" _l_ "'^ ^ — '^^'^ ^ ^^^^^ ~ '^^^ ~ 

2/3 dhi 2(3 dhi ' " exp (a;t + /3ft-n) + exp (a„: - /3ft,„) 

where rhi — {Si)j\i-i and exp(a^) = (exp(±/3 ^ J„i5'i)}Ar_i are expectation values of the 
— 1 spin system. Note that latter of these values can be rewritten as a ratio of two partition 
functions with fields hi ± J^i and with fields hi, respectively. With the remark that the next 
steps are only justified to the leading order in A^~^ one finds by a cumulant expansion = 
Jmrhi + JniJnjdrhi/dhj. This leads to eq.(|l^) with xi = JniJnjdrhi/dhj 

which can to leading order be approximated to xi — N^^^^dnii/ dhi due to the random 
character of the Jin- 

The theorem of Pastur ||] is central for the present work and in the remaining part of this 
appendix two technical aspects related to this theorem are worked out. Let Kij = kiSij be a 
non-random matrix in N dimensional space with all ki real valued and let Jij be a symmetric 
matrix (with Ja = 0), where the off-diagonal elements are independent random quantities 
with zero means and standard deviations N~^/'^. According to this theorem the resolvent 
R{z) = A^~^Tr{z — K + J}~^ satisfies the equation in the limit N ^ oo 

R{z) = A^-^Trjz - K - R{z)Y^ = N'^ ^{^-h- R{z)y^ . (22) 

i 

Focusing on the properties of the A^ -I- 1 solutions of ( p^ ) f{x) — N^^ jfci -I- x\ ^ is 
introduced and /ci < fc2 • ■ • < ^jv is presumed. Setting r — R{z — 0), the solutions of ( [2^ ) for 
the case z = are determined by — r ~ f{r). According to fig.(||) there are always A^ — 1 real 
solutions Ti (< — fci) for i = 2, . . .N . Depending on the values of h two cases are possible 
for the two remaining solutions: For case (i) both solutions are real and will be denoted by ri 
and To (with ri < t-q). For case (ii) a pair of conjugate complex solutions results with Im 
c_)_ > and with Im c_ < 0, respectively. 
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In the next step the hnear terms in z of the solutions Ri(z) of (g2|) are calculated by 
expansion for \z\ <C 1 near the real solutions r^. The ansatz Ri{z) — ri + ai z leads in 
linear approximation to — — ai z = f(ri + ai z — z) « /(r^) + f {ri){ai — 1) z from which 
a^ = f'{n){l + f'in)}-^ results. Fig.(|) shows, that -1 < /'(ro) < and /'(r,^o) < -1 
holds and in consequence ao < and a^^o > results. Thus with the secondary requirement 
Im R{z) > for Im z < 0, the solution of ( p^ ) is unique determined and given in the limit 
z ^ by ro in case (i) and by c+ in case (ii), respectively. The definition of R(z) satisfies 
this requirement as Imi?(z) = —N~^{lmz) {(A + Rez)^ + (Imz)^} holds, where A are 
the real eigenvalues of — K + J. Thus the solution of ( p^ ) together with the requirement (||) 
determines R(z) uniquely. 

Let us finally consider the special points A on the real axis in the z plane where the real 
solutions R(X) = f of (^2|) bifurcate to complex values. The values A and f are determined by 
the solutions of f = — /(r — A) and of f'{f — A) = —1. The first equation is just the theorem 
( p2| ) while the latter can be obtained by a discussion similar to fig.(|^). The definition of f(x) 
leads to the identity /(u) - f{v) + {v - u)f'{v) = N-^[v - ufY^t i^t + u)-^{h + w)"^ . 
Focusing to the original TAP case ki = I3~'^{1 — mf)~-^ + f3{l — (72) has to be used according 
to and (|2|). Setting u — — /3(1 — 92) and setting v — f — X the identity and the above 
equations lead to the important result A > . This result implies that all endpoints of the 
intervals with a finite p(A) are located on the positive axis. A 1/z -expansion of (||) shows 
that p(A) = for large negative A . Thus the minimum eigenvalue satisfies Xmin > which 
generalizes the result (^6|) to all values of x. 
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